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Abstract 

In a series of papers, G. Harmer and D. Abbott study the behavior of random walks 
associated with games introduced in 1997 by J. M. R. Parrondo. These games illustrate 
an apparent paradox that random and deterministic mixtures of losing games may 
produce winning games. In this paper, classical cyclic random walks on the additive 
group of integers modulo m, a given integer, are used in a straightforward way to 
derive the strong law limits of a general class of games that contains the Parrondo 
games. We then consider the question of when random mixtures of fair games related 
to these walks may result in winning games. Although the context for these problems 
is elementary, there remain open questions. An extension of the structure of these 
walks to a class of shift diffusions is also presented, leading to the fact that a random 
mixture of two fair shift diffusions may be transient to +oo. 
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1 Introduction 



The purpose of this paper is to study a family of random walks that include those arising 
in the games devised by J. M. R. Parrondo in 1997 to illustrate the apparent paradox 
that two 'losing' games can result in a 'winning' game when one alternates between them. 
We refer the reader to Harmer and Abbott (1999a,b), Harmer, Abbott and Taylor(2000) 
and Harmer, Abbott, Taylor and Parrondo(2000) in which Parrondo's paradox is discussed, 
large simulations of specific Parrondo games and mixtures thereof are presented and certain 
theoretical results are given. These authors also give a heuristic explanation of the paradox 
in terms of the Brownian ratchet, the original motivation for the suggestion of these games. 
Other references to the general subject are included in the above mentioned papers by Harmer 
and Abbott. The reader may also note the reference Durrett, Kesten and Lawler(1991) which 
also deals with the general question of showing that winning games can be formed by mixing 
fair ones. 

The suggested paradox may be visualized as follows. You are about to play a two-armed 
slot machine. The casino that owns this two-armed bandit advertises that both arms on their 
two- armed machines are "fair" in the sense that any player who plays either of the arms is 
assured that the average cost per play approaches zero as the number of plays increase. 
However, the casino does not constrain you to stay with one arm; you are allowed to use 
either arm on every play. You just tell the machine before beginning how many plays you 
wish to make. At the end of that number of plays, the machine displays the total amount 
won or lost. The question of interest in this context would be whether it is possible for the 
casino to still make money using only "fair" games. 

In this paper a random walk will refer to a Markov chain {S n : n = 0, 1, 2, ... } taking 
values in the integers, Z, which satisfies the discrete continuity condition 

\S n — S n -i\ — 1 a - s - f° r eacn n> 1. 
Let the transition probabilities for the random walk be denoted by 

Pj = P(S n+1 - S n = l\S n = j), qj = P(S n+1 - S n = -l\S n = j) 

and 

r j — 1 — pj — qj = P{S n +i = S n \S n = j) 

for j G Z. Assume that pjqj ^ for all j. For fixed integer m > 1, define a mod m random 
walk to be a random walk in which the transition probabilities p iy r iy q { depend only upon 
the congruence class mod m of the state i. Thus, these lattice regular or periodic random 
walks are such that for some specified integer m > 1, pj — Pj+m and qj — qj+m 

for all j e Z. 

More generally, define a mod m Markov chain on the integers to be one whose parameters 
depend only upon the congruence classes mod m of the states, namely, Pij = Pi+ m ,j for all 
integers This paper is concerned with the case of random walks, but places where the 
approach applies more generally are pointed out. 
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A mod m random walk is determined by the 2m parameters pj, qf, < j < m. Write 
P — (PojPi) ' ' ';Pm-i) with an analogous use of q to specify the walk's parameters. Observe 
that when m — 1 the walk is classical simple random walk, so our main interest is in the 
cases of m > 1. 

These random walks are viewed as games with the increment X n — S n — S n -i(n > 1) 
denoting the gain at the n-th play. We say that the game is a winning/losing/fair game 
according as the almost sure limit of S n /n is positive/negative/zero. 

For given m, write Z m := mZ = {km : k G Z} for the integer lattice of span m. In 
the games introduced by Parrondo, it is assumed that the transition probabilities depend on 
the state only to the extent that the state is or is not in Z m . Thus, Parrondo's games are 
characterized by 

(1-1) P{X n+ i = 1 | S , S u ... , S n ) = p'l[ Sn ez m ] +PMs n ?z m ] 

for some p, p' G [0, 1] and all n > 0. Write q = 1 — p and q' = 1 — p'. We may also write 
k = j mod m when k G j + Z m . 

For simplicity, we write G(m, p, q) to denote a general mod m random walk or game, 
but write G(m, p) for the game when each qj = 1 — pj (i.e. each Tj = 0) and write G(m,p,p') 
for the special Parrondo random walk or game satisfying ( |1 . 1| ) . 

The required notation and preliminary structure are introduced in the following section, 
in which the limiting results for G(m,p,p') games are given for illustration. The general 
case is covered in Section 3, while in Section 4 we resolve the central question about whether 
random mixtures of losing Parrondo's games can be winning ones. The asymptotic gain is 
derived in Section 5 while in Section 6 a certain expected interoccurrence time that appears 
in the previously obtained expression for this is also dreived. The method used to solve the 
recursion equations in these sections makes use of an extension of results of Mihoc and Frechet 
(cf. Frechet(1952)) that are provided in the Appendix to this paper. Continuous analogues 
to the random walks considered here are introduced in Section 7. These mod m diffusions 
have drift functions that are periodic step functions so that their embedded walks on the 
integers are G(m, p, q) walks. In Theorem 7.2 the drift rates under which the embedded 
walk has specified transition probabilities is determined. 



2 Preliminaries and Parrondo's Examples 

In the games suggested by Parrondo, the transition probabilities depend on the state only to 



the extent that it is or is not in Z m ; see (|1 . 1| ) above. The asymptotic behavior of these games, 
as for any mod m random walk is determined by that of its embedded walk on the lattice 
Z m . Since this embedded walk is equivalent to simple random walk, its asymptotics are well 
known and dependent solely upon a single parameter, the walk's probability of 'success'. In 
this section we introduce the notation required for the general case in Section 3 below, and 
illustrate the approach in the special case of a Parrondo G(m,p,p') walk by substituting in 
known results for simple random walk. 
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Let Ti <T 2 < ■ ■ ■ be the successive transition times of the embedded walk on Z m . That 
is Ti = min{n > : S n G Z m } and, for k > 1, 

T fc+ i = min{n > T k : S n - S Tk = ±m} 

with the minimum of a null set being defined to equal +00. Set T = 0. Write 

J n = m 1 Sr n+1 , £n = T n+ i — T n 

for n > so that {(J n ,£n) : n > 0} is a (possibly delayed) Markov renewal process (MRP) 
in which the embedded random walk { J n } is simply a classical random walk with constant 
probability of ' success', 

(2.1) p* m := P{J n+ i - J n = 1 I J n ). 

Hence, once p* m is known, the winning/losing/fair nature of the walk is easily determined. 
In general, for n satisfying T k < n < T fc+1 , 

mJ k+1 -2m _ S Tk+1 - 2m S n S Tk + m _ mJ k + m 
Tfc+i T k+ i n T k T k 

Thus, 

(2.2) m ( - -2-) <-<m(^ + l)/^. 
v ' \k + l k+lj'k+l-n- \k kj'k 

It is known for the classical random walk {J n } that J k j k converges a.s. as k — > 00 to p* m — q^, 
with q* m = 1 — p* m . Moreover, the stopping times {T k } are partial sums of iid r.v.'s having 
finite expectations so that 

T fc /A;^ J B(T 2 -T 1 )<oo. 
Upon taking limits in Q2.2Q one obtains that with probability one, 

(2.3) lim ^ = m ^-q* m ) 

n^oo n E(T 2 — Ti) 

Clearly then, this limit is 0, > or < according as p* m =, > or < q* m . 

The quantity p* m is evaluated for the general G(m, p, q) walk in Lemma |372| below. How- 
ever, for the special Parrondo G(m,p,p') random walk, the evaluation is immediate once we 
introduce the notation and approach that is needed for the general case, and so we give it 
separately here as 

Lemma 2.1 The Z m - embedded MRP of the G(m,p,p') random walk has transition proba- 
bilities determined by the 'success 7 probability 

(2.4) v* = 

^ ' " m plpm—l _|_ q/qm—1 

for all p,p' G [0, 1] satisfying \p — p'\ < 1. 
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Proof. The first part of this proof, through (|2.6|) below, is general and will be needed in 
Section 3. The rest is substitution of known results. 

Suppose J n = k. That is, for the original walk suppose St„ +1 = km. Since T n+ i is a 
stopping time, P(J n+ i — J n — 1 | J n — k) is just the probability that starting at So = 0, the 
random walk {S n } reaches m before it reaches — m. But Si equals 1 or —1 with probability 
p or q , respectively Thus if we let A denote the event that {S n : n > 1} reaches before 
it reaches mSi then the Markov property implies that p* m , the success probability for the 
embedded walk, satisfies the following recursion relation, in which we partition the event 
according to whether the original walk hits zero before m or not: 

(2.5) p* m = P(A)p* m + Po {l - P(A | Si = 1)}. 
Hence 

(2.6) p* m = Po P(A c \S 1 = l)/P(A c ). 

Since for the special case of this lemma, the conditional probabilities given 5*1 are just those 
that arise in the classical gambler's ruin problem, (cf Feller (1968, Chap. XIV) it is known 
that 



(2.7) P(A C | Si = 1) 



(qp m 1 — p m ) I (q m — p m ) if p ^ q 
1/m if p — | 



and P(A C \ Si = —1) is similar but with p and q interchanged. Substitution of (|2.7|) into 
(|2.6| ) now gives, when p ^ q, 



(2.8) P(A C ) = {q'q m ^ + p'p m ^){q - p)/(q m - p m ) 

and, therefore, P m is cis required by ( |2.4j ). When p — |, the substitution of ( |2.7|) yields 
Pm = P' ^° complete the proof. □ 



Note that by ( |2.4j ), p* m is the conditional probability that S n reaches m before — m given 
that So = and that the first m steps of S n are monotone. This structure is more readily 
seen in the general case of Lemma |3.2| below. 

For the special G(m,p,p') case the above result yields 

Corollary 2.2 (Harmer and Abbott(2000a)) When \p — p'\ < 1, the game G(m,p,p') is a 
fair, winning or losing game according as 

p r p m-l _ q r q m-l = 0> > g Qr < q 



The condition in Corollary |2.2| is more clearly expressed in terms of new variables x = p/q 
and y = p'/q', namely, the game G(m,p,p') is a fair, winning or losing one according as 

(2.9) y - aT^" 1 ) = 0, > or < 0. 
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Recall that the degenerate case q' = = p has been excluded. Since the inverse relationships 



are p = x/(l + x) and p' = y/(l + y), it follows from ( |2.9| ) that G(m,p,p') is fair if for some 
x > 0, p and p' are related as 

1 x ~m+i J 

g = 1 — p = and p' = — = -. 

1 + x 1 + x~ m+1 1 + x m ~ x 

Here are some examples. For x = 1, G(m, ~, |) is fair for every m > 1. For m = 4 and 
x = 2, the game (7(4, |, ^) is seen to be fair, and for m = 5 and x = 2, G(5, |, ^) is fair. 
When m = 3 and one chooses x = 3, one obtains the fair game G(3, |, ^). The associated 
games G(3, | — e, ^ — e) for a range of e > are the losing games used in the simulation 
study of Harmer and Abbott (1999a). The fact that these are losing games as indicated there 
is immediate from the following observation: If G(m,po,p' ) is a fair game, then G(m,p,p') is 
a losing game whenever < p' < p' and < p < po with p+p' < po+p'o', simply observe that 
p'/q' and p/q are increasing functions of p' and p, respectively, so that p' /q' = (po/%)~ m+1 
implies p'/q' < {p/q)~ m+1 whenever p' < p' and p < p . Since G(3, |, ^) is a fair game the 
result follows. 



3 General Mod m Random Walks 

Let {S n : n > 0} be a general (discretely continuous) random walk on the integers Z in the 
sense described in the Introduction above. The asymptotic behavior of {S n } can be described 
in terms of the two associated reflecting random walks on the negative and positive integers. 
The latter is obtained, for example, by replacing r and q by f = 1 — po and q = 0. It is 
known (cf. Feller (1968), Chap. XV.8 or Chung (1967), Sect. 1.12) that the corresponding 
reflecting random walk on Z + = {0, 1, 2, ... } is recurrent or transient according to 



(3.D E 



oo 

<?i<?2 • ■ ■ qi 
= oo 



or not. When one looks similarly at the reflecting random walk on Z" = {0,-1,-2, 
. . . }, the roles of the p's and g's are interchanged so that recurrence in this case holds 
if and only if 



(3-2) E 



oo 

P-lP-2 ■■■P-i 
= OO. 

i g_ig_ 2 • • • q-i 



Now return to the original walk on Z. The positive part of this walk, {5^"}, is a Markov 
renewal process in which all sojourn times are equal to one except those between successive 
visits to state 0. The distribution of these latter sojourn times is a possibly deficient mixture 
that includes with probability q the distribution of the first passage time from state — 1 
to state 0. The latter passage time is finite with probability one only if the reflecting 
random walk on Z~ is recurrent. Hence {S n } is a recurrent random walk if and only if both 
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reflecting random walks are recurrent, or equivalently, if and only if both ( ft.l|) and 
hold. Consequently, the walk is transient if and only if at least one of these series converges. 
Accordingly, the boundary of a transient random walk may consist of either or both of 
+00 and —00, depending upon which one or both of the series converge. (Cf. Karlin and 
McGregor (1959), Section 4 where the integral representations of the transition probabilities 
of the doubly infinite random walk are expressed in terms of those of the two corresponding 
reflecting walks.) 

Consider now, for fixed integer m > 1, a mod m random walk as defined in Section 1. 
random walk in which the transition probabilities pi, rj, depend only upon the congruence 
class mod m of the state i. (Note that when m = 1, the mod 1 random walk is just the 
classical random walk with constant transition probabilities.) Thus, for % = sm + / for some 
s E 7L and I = 0, 1, . . . , m — 1, we know that (p i5 r i5 qi) = (pi, ri, q{). Moreover, for s > 0, the 
summand in (B.1I) becomes 



, 33 n ■ ■ ■ qi _ Po f q qi ; • • q m -\ 1 " Qoqi ; ; ; Qi 

PlP2---Pi ?0 \ptiPl~ -Pm-l) PtiPl-'-Pl 

while for s < 0, a similar representation holds with the p's and g's interchanged. If we define 

POPI ■ ■ ■ Pm-l 



(3.4) p v 



Q0Q1 ' ' ' Qm-l 



then the divergence of (|3.1| ) holds if and only if p m < 1 while (|3.2|) holds if and only if p m > 1. 
By the above discussion, the walk is then recurrent, transient to +00 or transient to —00 
according; cLS Pm IS equal to, greater than or less than one. This then proves 

Lemma 3.1 For m > 1, a mod m random walk is recurrent, transient toward +00 or 
transient toward —00 according as 

(3.5) poPi ■ ■ -pm-i - q qi ■ ■ ■ q m -i = 0, > or < 0. 

It remains to evaluate p* m1 the probability of 'success', p*^, for the embedded walk on Z m . 

Lemma 3.2 For m > 1, and a mod m random walk G(m, p, q) wrt/j parameters 

P = (Po,Pi, • • • q = (?o, Qi, ■ ■ ■ , Qm-i) satisfying p t q t ^ for i = 0, 1, . . . ,m—l, 
one has 

f o a\ * PoPi---Pm-i Pm 

(3.6) p m = = . 

PoPl ■ ■ ■ Pm~l + <?0<?1 • • • Qm- 1 1 + Pm 

Proof. For this general case, set 

(3.7) w m = P(A C \S 1 = 1) and v m = P(A c |5i = -1). 
so that the expression for p* m in (|2.6|) becomes 

P„ = PoVm(poV m + qoV m y l 
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m = 5- 




Figure 1: An illustration for m = 5 of the correspondence between first hitting paths from 1 
to 5 and those from 4 to 0. (Probabilities in parenthesesare those for the indicated segments 
on the reversed path.) 



Thus (|3.6|) will be proved once it is established that 

^fy Vm = V\V2 ■ ■ -Pm-1 

V m <?1<?2 • • • q m -l ' 

By definition, v m (v m ) is the probability (of 'ruin') that starting at 1 (—1) the random walk 
reaches m (— m) before it reaches 0. Moreover, by the modulo structure of the walk, v m is the 
same as the probability that starting at m — 1, the random walk reaches before m. Thus, 
v m , for example is the same as /im m the usual notation for these taboo probabilities; cf. 
Chung (1967, Sect. 1.12) where these are derived for the random walk. Direct substitution of 
these exact values would then justify (|3.6|) . Since we only require the ratio of these two taboo 
probabilities, the following mapping approach suffices, and may be of separate interest. 

We first construct a 1 — 1 correspondence between the set, 1^, of paths that go from 1 
to m without hitting and the set, Gk, of paths that go from m — 1 to without hitting m. 
Thi s correspondence is a simple reversal: If s fc = (si, s 2 , ■ ■ ■ , Sk, m) denotes a path in T k so 
that si — 1, 5^ = m — 1 and 1 < < m — 1 for 1 < i < k, the corresponding reversed path 
in Gk is 

tfc(sfc) = tfc = (ti, t 2 , ■ ■ ■ , tk, 0) = (sfc, Sfc_i, ... , si, 0). 

(The reader can visualize the reversal of a path in the illustration of Figure 1. In fact, the 
result becomes fairly transparent once one recognizes the effect on paths of flipping the time 
axis.) 

For a given path s^, let (z/~) equal the number of transitions from % to % + 1 (i to 
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% — 1). Then 



P {{Si, S2, ■ ■ ■ , Sfc+i) — Sfc|Si — 1) 

(3.9) 

with the last step following since = 0, z^_i = 1 and v^, x = v£ — 1 > for 1 < i < m — 2. 
For the reversed path tfc(sfc), where an 'up' transition of Sj to Sj + i in becomes a 'down' 
transition of Sj + i to Sj. Write vf and z/~ for the corresponding numbers for t& so that 

(m— 1 \ m— 1 
nto-i)^" 1 - 
i=l / j=2 

But it is clear from the correspondence that dj = v^_ x . Thus for every k > m — 1 and every 
path s k e T k the ratio of ( |3.9|) over (|3.10|) , namely pip 2 ■ ■ -Pm-i/<li<l2 • • ■ ?m-i> is constant. It 
now follows immediately that ( |3.8|) holds, thereby completing the proof. □ 

4 Random Mixtures of Parrondo Games G(m,p,p') 

The main question of interest for these games concerns what happens to a player's fortune 
when two or more games are played in some alternating fashion. For example, if two different 
games are known to be fair, can a player create a winning game by randomly choosing 
between the two at each play? Observe first of all that for tt e [0, 1], the random mixture 
of two games, G{m, p, q) and G{,m, P, Q), in which at each play the former is chosen 
with probability 7r,is also a mod m game, namely, G(m, irp + (1 — 7r)P, 7rq + (1 — tt)Q). 
Since Lemma 3.1 characterizes the winning or losing nature of any such game, the question of 
whether the random mixture of two fair games is a winning game or not has been theoretically 
answered. By the way, the criterion in Lemma 3.1 implies that if p and q are interchanged 
in a fair game G{m, p, q), it remains fair, whereas a losing game would be turned into a 
winning game. Moreover, the nature of the criterion is such that it should be the exception 
rather than the rule for a random mixture of fair games to remain fair. Thus at this stage, 
the existence of fair games whose mixture is winning (or losing) would appear to be less 
paradoxical. 

A couple of general questions of interest are as follows. Suppose we say that two fair 
games, A and B, are mutually supportive if any other game consisting of a sequence of plays 
of game A or B is not a losing game whenever the game choices are made independently of 
previous outcomes. Do mutually supportive pairs of distinct games exist? Is it true that if 
a non-trivial random mixture (in which game A is chosen independently at each stage with 
constant probability) does not result in a losing game, then the two games are mutually 
supportive? 



m—l 

n pi* & 

1=1 

f m— 1 \ m—2 

n pi ) n 

i=i j j=i 



wl-l 
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In this section, we give a complete answer to the structure of random mixtures in the 
special case of the Parrondo game, G(m,p,p'). Although this is done by rather elementary 
methods, more general questions involving mixtures appear to be quite difficult. 

Consider the random mixture, G(m, irp + (1 — iv)(3, np' + (1 — tt)/3'), of the two Parrondo 
games G(m,p,p') and G(m, (3, /?'), in which the mixing probability is n G (0, 1). Set 

(4.1) x = p/q } y=p'/q', x = — Q — , y 



1-/3' " 1 -/?' 



and 



(4.2) x- ^P+t 1 -^ _ P + W y P' + W 



{np + (i _ q + \(i-py * q ' + x(i-(3'y 

where A = (1 — tx)/ti. Assume without loss of generality that (3 < p, or equivalently, x < x. 

The question to consider is whether the random mixture of two losing games can be a 
winning game. Suppose first that the two given games are fair. That is, by Corollary |2?^ in 
the form (|2.9|), our question is whether it is possible to have 



(4.3) y = x - m+1 , y = x- m+1 and y > x~ m+1 . 

For simplicity, write m — 1 = r so that r = 1, 2, • • • . Simple algebra leads to 

_ = x(l + x) + Xx(l + x) _ = y(l + y) + Xy(l + y) 
^ ' ' ' ' 1 + A + x + Ax ' V 1 + \ + y + \y 



Substitution of the first two equations of ( (4.3| ) into y permits the inequality y > x r to 
be written after simplification as 

/Ar , 1 + A + F + Ax r (l + A + x + Aa;) r 

(4-5) ttt^ > 



(1 + X)(xx) r + Xx r + x r ((1 + X)(xx) +Xx + x) r ' 

Clearly, this can never hold if r = 1, (i.e m = 2). We assume, therefore, that m > 2 in the 
remainder of this section. 

If one introduces functions f(a) = a r and g(a, b) = (1 + A + a + A6)/((l + X)ab + Xa + b), 
then (|4.5|) involves a form of inverse composition, namely, 



9(f(x),f(x)) > f(g(x,x)). 
On the other hand, (|4.5|) may be written equivalently in terms of 7r as 



, , 1 + TTX" + (1 - 7l)x r 1 + 7lX- r + (1 

( 4 - 6 ) TV—^TTT, "TTw > 



(1 + -KX + (1 - ir)x) r 1 + -nx- 1 + (1 - vr)x- 1 ) r ' 

Thus, this inequality is one about norms on the simplex as may be seen as follows: If we set 
u = (l,x,x) and v = (1, 1/x, l/x), (|4.5| ) is equivalent to 



MlWlMk/i > ll v IIWl|v||i. M , 
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where the norms are with respect to the measure \i that assigns masses 1, tt, 1 — tt to the 
coordinates 1,2,3, respectively. [In this context, the special case of x = 1, in which the first 
game is the classical fair random walk, (and which is the case relevant to the examples in 
Harmer and Abbott (1999a)), is describable as a comparison between the r-norms of the 
ray projection onto the unit simplex of the vectors (1, l,x) and (1, l,x _1 ) (or equivalently, 
(l,x,x). Moreover, in the case of purely random mixing (tt = 1/2), the inequality is more 
enticing in that it may be stated as above but for vectors (1, 1, 1, x) and (1, 1, 1, x _1 ) under 
counting measure on the coordinates.] 



Fix x = a > 1. By cross multiplying in (|4.6| ), the inequality is equivalent to the positivity 
of the polynomial 

Q(x) : = (1 + A + a r + Xx r ) ((1 + X)ax + Xa + x) r 

-((1 + X)a r x r + Xa r + x r )(l + X + a + Xx) r 
= (1 + A + a r )((l + X)ax + x + Xa) r - Aa r (l + A + a + Xx) r 

+x r {A((l + X)ax + x + Xa) r - ((1 + X)a r + 1)(1 + A + a + Xx) r } 

' ' {(1 + A + a r )((l + A)a + iy(Xa) r - J - Aa r (l + A + af^X 1 }^ 



E 



r 

- r 



{A((l + A)a + l) k (Xa) r - k - ((1 + X)a r + 1)(1 + A + a) r ~ k X k }x r+k 



k=0 

>2r 



Upon writing Q(x) = Ylfj=o ( li x ' 1 \ ^ follows that the coefficients are 
(4.7) 

( (pa r {(l + A + a r )(l + A + a-y X r ~ j - (1 + A + a) r ^X j+1 } for < j < r, 

qj=\ (l + A + a r )((l + A)a + l) r -((l + A)a r + l)(l + A + a) r for j = r, 

{ ( i I r )a r {(l + A + a- 1 ) J '- r A 2p -J' +1 - (1 + A + a - r )(l + A + a) 2r ^X^ r } for r < j < 2r. 

Since the expressions within the parentheses in the first and third cases are increasing in 
j, there can be at most one change of sign among the first r coefficients and at most one 
among the last r. Thus, regardless of the sign of the middle coefficient, q r , there are at most 
three changes of signs in the coefficients of Q with the exact number depending upon the 
signs of go, Qr-i, Qr-Qr+i, <?2r- (One may check that q<i T is always positive for a > 1, while go is 
negative when A < 1 (i.e tt > l/2i) or when a < 1.) By Descartes's rule of signs, the number 
of positive roots of Q{x) = does not, therefore, exceed 3. 

It follows directly from the definition of Q that Q(a) = 0. However, one may check that 
x = a is in fact a double root for all positive a. To see this, compute from (f|), 

Q'( x ) = rx r - 1 A((l + X)ax + x + Xa) r 
. v +r((l + X)a + 1)(1 + A + a r + Ax r )((l + X)ax + x + Xa) 1 - 1 

^ * -((1 + X)a r + l)rx r -\l + X + a + Xx) r 

-rA((l + X)a r x r + x r + Aa r )(l + A + a + Aa;)^ 1 
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so that after simplification 

Q'(a) = r{l + \ya r - 1 {a + l) r - 1 {\a r {a + l) + {{l + \)a + l){l + a r ) 
-((1 + X)a r + 1)(1 + a) - Xa(a r + 1)} = 

for any a. Since this implies that x — a is a double root of Q, it follows from Descartes's rule 
of signs that Q has either two or three positive roots. In either case, we need to know that 
the root at x = a is the largest positive root. To show this, differentiate (|4.8|) to obtain 



r - l Q'\x) = A(r - l)a; r - 2 ((l + \)ax + Xa + x) r + 2Arx r " 1 ((l + X)a + 1)((1 + X)ax + Xa + x)^ 1 
+ (r - 1)((1 + A)a + 1) 2 (1 + A + a r + Ax r )((l + X)ax + Xa + x) r ~ 2 
-((1 + X)a r + l){(r - l)x r - 2 (l + A + a + Xx) r + 2Arx r ~ 1 (l + A + a + Ax)"" 1 } 
-A 2 (r - 1)((1 + X)a r x r + x r + Aa r )(l + A + a + x) r " 2 

from which 

Q"(a) = r(l + A) r ~V" 2 (a + l) r ~ 2 {A(l + A)(r - l)a r (a + l) 2 + 2Ara r ((l + A)a + l)(a + 1) 
+ (r - 1)((1 + X)a + 1) 2 (1 + a r ) - (1 + A)(r - l)(a + 1) 2 ((1 + X)a r + 1) 
-2Ara(a + !)((! + X)a r + 1) - A 2 (r - l)a 2 (a r + 1)}. 



By grouping the terms within the parentheses here according to powers of a, this becomes 

Q"(a) = (1 + X) r ~ 1 ra r - 2 (a + l) r ~ 2 {A(r - l)(a r+2 - 1) + 2Xr(a r+1 - a) + A(r + l)(a r - a 2 )}. 

Thus, for r > 2 (m > 3), Q"{a) is positive, negative or zero according as a > 1, a < 1 or 
a = 1. This implies in particular that when a = l,x = lisa triple root, and hence the only 
root by Descartes's rule of signs. Thus, when a = 1, x = 1 is the only positive root, insuring 
that Q(x) > for all x > 1. For a > 1, the fact that Q"{a) > shows that this double root 
at x = a is a local minimum. Since by ( [4.7| ) the leading coefficient, q2 T , is positive for all 
A and all a > 1, this insures again that x = a is the largest real root of = 0, thereby 
establishing that Q(x) > for all x > a whenever a > 1. This completes the proof of 

Theorem 4.1 T/ie random mixture, G(m,TTp+(l — 7c)j3,Trp'+(l — 7c)j3'), of two fair games, 
G(m,j3,/3') and G(m,p,p') is a winning game whenever m > 3 and ^ < (3 < p < 1. 

Corollary 4.2 There exist losing games, the random mixture of which is a winning game. 
Proof. By Corollary |2.2|, the expression whose sign determines whether a game is winning, 



losing or fair, is a continuous function of its variables. It is therefore clear that for the games 



appearing in the statement of Theorem |4.1] , one may make a sufficiently small change in the 
parameters (/?,/?') and (p,p') to make the associated fair games become losing ones, while 
preserving the inequality that ensures that the random mixture of the two remains a winning 
game. □ 
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The example presented in Harmer and Abbott (1999a) may now be described as follows. 
Take m = 3, /3 = | = /3', p — | and p' = ^. The games G(3, |, |) and G(3, |, are 
fair by Corollary [2.2| , so that by Theorem [4.1| , the mixture G(3, |, ^) is a winning game. 
Consider now the games used by these authors, G(3, | — e, ^ — e) and G(3, \ — e, \ — e), and 
their random mixture G(3, | — e, ^ — e). It is clear that the first two are losing games for 
each positive e < 1/10 and that there would be some positive value 6q < 1/10 for which the 
mixture remains a winning game whenever < e < Eq, as postulated in Harmer and Abbott 
(1999a). 

In this section we have considered the random mixing of two G(m, p) walks. One is 
also interested in deterministic mixtures. Simulations in Harmer and Abbott (1999a) indi- 
cate that deterministic mixtures of the two games proposed by Parrondo turn their separate 
losing nature into a winning combination. It is difficult in gneral to analyze such determin- 
istic mixtures since it requires computing the stationary probabilities of the product of the 
associated stochastic matrices. To expand upon this, suppose one has two distinct G(m, p) 
games called A and B with parameters aj, 0, 1 — a 3 - and bj, 0, 1 — bj, respectively. By Lemma 
3.2, the probabilities p^(A) and p* m (B) for the two games would equal 1/2 (i.e., the games 
would be fair) if and only if 

- 1 m-1 , 

a,- _ -n- bj 



^ II. II. 



.5. l-} n 

j=0 3=0 J 

Consider now the random walk formed by alternating the transition probabilities of these 
two. Then the two-step process is also a random walk, though one with jumps of two units 
and with non-zero probabilities of zero jumps. That is, the alternation of two G(m, p) 
games is a G(m, p, q) game. This 2-step process is then reducible with two classes, the odd 
and the even integers. If the walk starts in state '0', for example, the corresponding quotient 
of relevant parameters is 

^ 4 1Q v (a Q bi)(a 2 b 3 ) ■ ■ ■ (a m - 2 b m -i) 



(1 - a )(l - &i) • • • (1 - a m _ 2 )(l - 6 m _i) ' 

Since only half of the parameters enter here, it is clear that this ratio may be greater or 
less than or equal to 1 even when the separate games are fair. This implies that when m 
is even, the alternation of two fair games may be either fair, winning or losing. Notice that 
even if one imposes the natural restriction that a fair game must be fair for all sarting states 
one gains nothing more since, for example, the condition for fairness starting in state 'V , 
namely, 

(a 1 6 2 )(a 3 6 4 ) • • •(a m _i& ) 



[l-ai)(l-6 2 )..-(l-a m _ 1 )(l-6 



is equivalent under (|4.9| ) to the expression in ( |4.10| ) being set equal to 1. 

When m is odd, the alternation of fair games is fair as can be seen by considering the 
two-step game as a mod 2m game for which fairness requires by Lemma 3.1 that the product 
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of ( |4.10| ) and the following displayed quotient be equal to 1, which follows from ( |4.9| ). Thus 



the alternation of these fair games cannot result in winning ones when m is odd. 

The story is different, however, for [AAB.B], the mixture in which two plays of game 
A are alternated with two plays of B. In view of the previous paragraphs, this game is 
equivalent when m is odd to an alternating [AB] game but one in which both A and B are 
G(m, p, q) games. For m = 3 this is reasonably tractable. In particular, if one of the games 
is the classical simple random walk one can show that the mixture is indeed a winning game 
under a natural restriction on the second game. For the special case of AABB in which A 
and B are the fair games G(3, |, |) and G(3, |, ^) corresponding to Parrondo's example, 
one can show that the asymptotic average gain is 0.0218363 > 

5 Direct Calculation of the Asymptotic Expected Av- 
erage Gain for a G(m, p) Game 

By (|2.3|), since p* m has been evaluated, the asymptotic average gain (or loss) would be known 
once E(T2 — T%) is computed. A closed form for this expected inter-occurrence time is 
discussed below since it is of interest in its own right for these processes. However, the 
asymptotic average gain, lim^oo S n /n, being a limit of bounded r.v.'s, may also be derived 
directly by obtaining the limit of the corresponding expectations. We do this as follows. 
Consider the game G(m, p). Define 

(5.1) /4. j) : = E(S n+k - S n \S n =j mod m) 

k 

= EQ2 E ( X i\ S o=J modm), 

emphasizing by the notation the fact that the expectation depends only upon the congruence 
class of S n modulo m and not upon the actual value of S n nor of n. In fact, the random 
walk S n is equivalent to the random walk on the circular group of integers mod m where 
a positive move is taken to be in the clockwise direction. Clearly, 



/4°i = Po(l + ^ 1) )+go(-l + ^- 1) ) 
Similarly, for j — 1, 2, . . . , m — 



(1) . (m-l) 

Pa-qo+PoVk +QoHk ■ 



(5.2) = Pj - q 3 + Pl ^ +1) + 

n) 

form, write /J, k = \ ■ ■ ■ , [f k )' and b = (p — go, Pi — Qi, ■ ■ ■ ,Pr — Qr)' as m x 1 column 



where we equate /z^ = ^ an d a4 ^ = /^i" 1 ■ To express this conveniently in matrix 
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vectors and set 



(5.3) 



C 



p 
qi pi 
<? 2 p 2 



go 
o 





g r _i p r _ 
p r q r 0. 





The reader should note that if {S n : n > 0} were a more general mod m Markov chain, 
the vector b would be given by 



That is, the transition matrix C for the Markov chain of congruence classes of {S n } is formed 
from the original chain's transition matrix P by summing over all states in the appropriate 
congruence class. With these defmtions, the limit of ( |5.4j ) applies to a general mod m 
Markov chain. We shall continue, however, with the G(m, p) case in order to obtain explicit 
values. 

The value of this limit depends upon the periodicity of C. Suppose first that m is odd. 
In this case, C is an irreducible aperiodic stochastic matrix provided only that pjqj ^ for 
each j. Thus the limit exists and is a stochastic matrix, each of whose rows is the row vector 
of stationary probabilities associated with C, 7r = (7r ,7ri,... ,7r m _i), say. It is a known 
result of G. Mihoc (cf. Frechet (1952), pp. 114-116) that the entries in tz are proportional 
to the diagonal cofactors of I — C. (See Appendix A below for this and other results to be 
used below.) 

Let 7j m denote the (i, i)-th cofactor of I — C. These are tractable for reasonable values 
of m. Due to the cyclic structure underlying the matrix C it is necessary only to obtain the 
first cofactor for each m. The first few values are: 

(5.5) 713 = l-p x q 2 , 714 = 1 - Pi<?2 - P2<?3, 




3 



and C would be determined by 



Cij = P(S n+ i = j mod m\S n = i mod m) 




715 



1 - Pl<?2 - P2<?3 - P3<?4 + Pl<?2P3<?4 
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and 



716 = (1 - Pl<?2 - P2&)(1 - P3<?4 - Vi%) - P2P3Q3Q4 

= 1 - £>1<?2 ~ P<iq.Z ~ Pzq\ - P4?5 + Pl<?2P3<?4 + P\q2P^ + £>2<?3P4g5 

The remaining diagonal cofactors are then obtained for each m by successively applying the 
cyclic permutation of (po,pi, • • • ,Pm-i) m to (pi,P2, ■ ■ ■ ,Pm-i,Po)- For the case of a Parrondo 
G(m,p,p') game with m — 3, the situation studied in Harmer and Abbott (1999a), ( |5.5| ) 
implies that 

7i 3 = 1 - pq, 723 = 1 - pq', j 33 = 1 - j/g. 
A general formula, presumably known, is possible for these cofactors, namely, 

m-2 

7lm = 1 - X] + PiVi+lPjVj+l 

i=l l<i<j-l<m-3 

(5.6) - X Piqi+iPjqj+iPkqk+i h — 

l<t<j— 1<*— 2<m— 4 

with the series continuing as long as the largest subscript does not exceed m — 1. Thus for 
I = [{m — l)/2], the last term has sign (—1)' and involves I subscripts h, . . . ,ii satisfying 

1 < i\ < %2 — 1 < is — 2 < • • • %i — I + 1 < to — I. 



As indicated by its appearance, (|5.6j ) follows from an inclusion-exclusion argument based 
on the number of pairs of adjacent diagonal l's used in the evaluation of the cofactor's 
determinant. (All diagonal cofactors are of course equal for each value of m > 3 whenever 
the parameters pj and qj do not depend on j.) 

As mentioned earlier, the stationary probabilities associated with C are proportional 
to these diagonal cofactors so that in our previous notation 7Tj = 7i+i, m /7. m where j. m = 

'Jim ~\~ ' ' ' ~\~ ^irnm- 

An early reference for the study of the general cyclical random walk on the integers 
modulo m, the one whose transition matrix is C, is Frechet ((1952), pp. 122-125. This is 
in effect a 1938 reference for this random walk, called by Frechet, "mouvement circulaire", 
since the material is present in the 1938 first edition of his book. He works out as an example 
the stationary probabilities for the case of to = 4. He obtains 714 as P2P3 + <7i<?2 which is 
easily seen to agree with the expression given above in ( p.5| ). 

The asymptotic average gain given by ( |5.4j ) now follows directly from the above for the 
case when m is odd. It is of the form A m (l, 1, . . . ,1)' with 

^ m 

(5.7) \ m = 7T m b = ~ gj-l)- 

7-m i=1 
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Consider now the case of m even, say m = 2k for k > 2. Then C is the stochastic matrix 
of a periodic Markov chain of period 2. By clustering the even and odd rows and columns, 
it may be written in the form 



(5.8) C 



A 
B 



/ AB 


° ) 


,C 2s = ( 


\ o 


BA ) 





in which A and B are k x k stochastic matrices. Consequently, 

{AB) 8 \ 2s+1 _ / A(BAY 
(BA) S J ' ~ \ B(AB) S 

in which both AB and BA are irreducible aperiodic recurrent stochastic matrices. If 8, p 
represent the vectors of limiting stationary probabilities for AB and BA, respectively, and 
if D and R are the matrices all of whose rows are 8 and (p), respectively, then 

lim C 2s = ( D ° 1 lim C 2s+1 = ( ° R 

and so ( |5.4| ) becomes in the case of m even, 

(5.9) \im»Jn= 1 -(° D b. 

By the result of Mihoc, the elements of the common rows 8 and poi D and i? are proportional 
to the diagonal cofactors of AB and BA, respectively. However, as shown in the Appendix 
below, the diagaonl cofactors of I — C are made up of those of l m /2 — AB and l m /2 — BA and 
that the column sums of the latter are equal and equal to 1/2 of the sum of the diagonal 
cofactors of I — C; cf (|A.6| ) below. In view of (|5.9| ) it follows that (|5.7| ) holds true as well 
when m is even. We summarize this as 

Theorem 5.1 For the general G(m,p) game, with probability one, 

S 1 m 

(5.10) lim — = A m = 7r m b = V^mfe-i - Qi-i)- 

n^oo n 7. m ^ 

in which the ji are the diagonal cofactors ofl — C and j. m is their sum. 

For the special case of a G(m,p,p') walk, the limit of interest in (|5.10Q becomes 

Am = {(p' -q')llm + {p-q){l- m -llm)}h-m 

(5.11) = 2p-l + 2(p / -p) 7lm / 7 . m . 
From ( |5.5| ), the first few values of 7. m for a G(m,p,p') walk are 

7.3 = 3 — pq — pq' — p'q = 2 + p'p 2 + q'q 2 

7 . 4 = 4 - 4 pq - 2pq - 2p'q = 2(1 - pq) + 2(p'p 2 + q'q 2 ) 

7.5 = 5 — 9pq — 3pq' — 3p'q + pq(pq + 2pq' + 2p'q). 
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For Game B of Harmer and Abbott (1999a), in which m = 3, p = 3/4 — e and p' = 1/10 — e, 
one obtains 

713 = 13/16 - e/2 + e\ 7 . 3 = ^ - \ + 3 £ 2 , 

80 5 



from which the limit in ( 5.11| ) becomes 



147 - 24e + 240e 2 9 9N 

5.12 A 3 = -2e r = -1.74e - .16e 2 + 0(e 2 ) 

v ; 169 - 165 + 240e 2 V ; 

This value appears to differ from the one implied by the simulated curve for Game B 
shown in Fig. 3 of Harmer and Abbott (1999a). The value for the curve given there for 
n = 100 is approximately —1.35, whereas for e = .005 and n = 100, the value from (|5.12 ) 



is approximately n\ 3 = —1.74/2 = —.87. The difference is that the slope of the simulated 
curve is affected by the early transient behavior; in a private communication, Harmer and 
Abbott confirm the agreement with this theoretical limit of their simulated slope when the 
first 100 plays are excluded. The analogous value for their Game A (where p — p* — | — e) 
is n\s = {—2e)n = — 1 which agrees with the curve for Game A given in their Fig. 3. 

For the randomized game that chooses between Games A and B with probability 1/2, 
one obtains p = | — e and p' = ^ — e for which 

49 e 2 709 e n 2 

7i3 = 1- £ , 7-3 = 1- 3e . 

' 64 4 ' 320 10 

Thus in this randomized case the asymptotic slope of S n /n is by ( |5.11| ) 
1 13 x 49 f 52 x 6111 n , 9s 

the expansion used in the first step requires only that e < .876. For the parameters n = 100 
and e = .005 of Fig. 3 of Harmer and Abbott (1999a) the asymptotic approximation becomes 
nX^ = 2.54 — .98 = 1.57. This differs from their simulated value of about 1.26, again due to 
early outcome effects. The reader might note that the graphs in the insert of Fig. 3 seem to 
be closer to those of ( |5.12| ) and ( p,13| ). 



As an illustration for even m, consider m = 4 for which the matrices become 

A=( P ' q ' V B=( q P 
\ q p J \p q 

and 

AB = 

Hence 5 U = (1 — 2pq)(l — 2pq + p'p + q'q)^ 1 and thus 

'^3 _ „/„3^ 



p'p + q'q —p'p — q'q 
2pq — 1 1 — 2pq 



(p' — p)(l — 2pq) 2{p'p — q'q 



( 5 - 14 ) A 4 = / 9 , \ , , —+p-q 



p'p 2 + q'q 2 + 1 — pq p'p 2 + q'q 2 + 1 — pq ' 
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see also ( |6.5| ) below. 

In this section, we restricted consideration to G(m, p) games. The approach applies as 
well to G(m, p, q) games but with the simplifying zero diagonal of C being replaced with 
the r/s. 



6 Expected Interoccurrence Times of Visits to Z m 



Set Tj = E(Ti\Sq = j) for j = 0, ±1, . . . , ±(m — 1) to denote the expected time of the first 
visit to Z m of a G(m, p) walk {S n } starting at j. In the expression ( |2.3| ) for the asymptotic 
average gain, the denominator E(T 2 — Ti) is equal to r . Hence, an alternate derivation of 
the asymptotic average gain would be, in view of Lemma ( |3.2| ), to derive r . This may be 
done by solving the recursion relations satisfied by the r/s, namely, 



(6.1) 



T j = Pj T j+i + 1j T j~i + !> for 3 = 0, ±1, • • • , ±(m - V 



with boundary conditions r_ 



r m = 0, where for negative j we have pj = Pj +m and 



qj = qj+m for a mod m walk. The solution of (|6.1|) is given for example in Chung(1967, 
1.12.(8)) in which the reader should note that the p^'s in this reference are related to the 
reciprocals of those used here. 

The expression that one obtains in this way is quite complicated even in the case of 
m = 3 and difficult to simplify into the more tractable expressions that can be obtained by 
direct solution of ( pM[ ) by matrix inversion. For if r := (r m _i, ... , 7~i, r , r_i, . . . , r_ m +i)' is 
the (2m — l)-dimensional column vector of expected occurrence times, 1 is the (2m — 1)- 
dimensional column vector of ones and G denotes the (2m— 1) x (2m— 1) matrix of coefficients 
in ( |6.1| ) then the system ( |6.1| ) may be expressed as r = Gr + 1 whose solution, with H = I— G 
is expressible by 



(6.2) 



(I-G)- 1 ! = ET 1 !. 



Thus the expected interoccurrence times of Z m are given as the row sums of the matrix H _1 . 
The matrix H whose inverse is needed is a Jacobi matrix with — p^s below a diagonal of l's 
and — qiS above it, namely, 



-Pr, 



-q m -x 

-2 1 —qm 












1 

-Po 




1 

-Pm-l 



-qo 



o 



o 



o 



o 
o 
o 



~P2 1 -<?2 

-Pi 1 
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In particular, by ( |2.3| ) the required quantity, E(T 2 — T x ) = r , in the computation of p* m , is 
the sum of the middle row of H _1 . Thus, if H it j denotes the {i, j}-cofactor of H = I — G and 
|H| denotes the determinant of H, then r = iJ. TO /|H| where H. m = H\ m + • ■ • + H 2m _^ m . 
When m = 3,Hisa5x5 matrix whose middle cof actors are straightforwardly shown to 

be 

#13 = PiPo(l -PiQ2), H 23 = Po (l- Pl q 2 ), H 33 = (1 -piq 2 ) 2 , 
H43 = go(l-Pig2) and H 53 = q q 2 (l - Pl q 2 ). 

Hence 

H. 3 = (1 - pi<? 2 )(3 - Piq 2 -P2qo~Poqi) 

and 

|H| = (1 -piq 2 )(i -piq 2 -p 2 qo - Mi) = (1 - pm) (poPip 2 + qoqiq 2 )- 

Therefore, for m = 3, 

r = E(T 2 - Tx) = 1 + 2/ (p pip 2 + q qiq2)- 
By (p.3|) and fl2.4|), this implies that with probability one, 



(6.3) A 3 = lim ^ - 3{PoPlP2 ~ qoqiq2) 



n 2 + p pip 2 + q Q qiq 2 

The reader may check that this agrees with the expression given for A3 in ( 5.10| ). 

For m = 4, G is a 7 x 7 matrix, and the middle column's cofactors for the corresponding 
HI = (I — G) -1 are easily computed to be 

Hia = p pip 2 \K\, H 24 = p pi\K\, H u = p Q (l -p 2 q 3 )\K\, 

H U = \K\ 2 , H 5 , = q (l - Pl q 2 )\K\, H u = q q 3 \K\ 

H74 = q q 3 q 2 \K\. 

where K is the upper left ( and lower right) (m — 1) X (m — 1) corner matrix of HI. This 
gives 

H 4 = \K\ [3 - Po qi - Piq 2 - p 2 q 3 - p 3 q + (pi - q 3 )(p 2 - q )] 
and, by expansion along the middle column, the determinant of HI is 

|H| = |K|[p piP2P3 + qoqiqm)- 

Therefore, after simplification, 

EI-4 2(p Pi + P2P3 + <?0<?3 + ?2<?l) 



(6.4) r 



|HI| P0P1P2P3 + qoqiq2q3 
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So that by ( |2.3| ) and ( |2.4| ) the asymptotic slope of the random walk for m = 4 is 
(6 5) A 4 = lim — = 2 ( poPlP2P3 ~ gogigaftQ 

with probability one. This is consistent with the result obtained by the methods of Section 
4; see (4.10). 

The above discussion focuses on G(m, p) games rather than the more general G(m, p, q) 
games. Only minor modifications for the latter are needed. The term TjTj is added to the 
right hand side of the equations (|6.1| ). This results in a substitution of Pj/{pj + qj) and 
1j/(Pj + Qj) f° r the parameters of the walk, and, more significantly, a replacement of the 
vector I in the solution (|6.2| ) by the vector of the reciprocals, pj + qj- A benefit of working 
out the more general case would be that whenever m is even, one could reduce the problem to 
one of order m/2 by observing that the embedded walk on Z m is equivalent in its asymptotic 
behavior to that of the 2-step random walk in which the parameters would become the 
products, PoPi, <?o<?m-i; etc- One can see this already in the example of m = 4 above, which 
the reader may compare to the case of m = 2 for the associated 2-step case. 



7 A diffusion analogue of a general random walk 

Partition the real line into intervals Jj = + 1] = j + (0, 1], for j = 0, ±1, ±2, ■ • ■ . Let 
H = {jij : j = 0, ±1, • ■ ■ } be given constants. For real x set 

(7.1) n(x) =^2v j l J .(x). 

j 

Now define a diffusion {W t : t > 0} in terms of a standard Brownian motion {B t : t > 0} by 

(7.2) dW t = dB t + fi{W t )dt, 

for t > 0. For this process, introduce the probabilities of transition between consecutive 
integers, namely, 

(7.3) pj = Pj(fij, A^-i) = P[W. hits j + 1 before hitting j — l\W = j] 

and let qj = 1 — pj. Observe that qj(jij, Hj-i) = Pji^^j-i, by reflection. 

To obtain expressions for the pj in terms of the pertinent drift rates, \Xj and fij-i, we will 
use the scale function of the diffusion. For this, fix constants a < b and define for x e [a, b] 
the first passage probabilities 

(7.4) u(x) = P[W. hits b before a\W = x]. 

The backward equations for the Markov process W. imply that u satisfies the second order 
differential equation u" + 2/x-u' = 0, the solution of which is of the form 

u(x) = c / exp{— 2 / fi(z)dz}dy + b. 

J a J a 
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The boundary conditions, u(a) = 0,u(b) = 1 then give 

J* ex P If v(z)dz}dy 



(7.5) 



u(x) 



f a exp{-2 fln{z)dz}dy 



Note that in the case of fij = \i for every j, this becomes the formula of Anderson(1960, 
Theorem 4.1); for a < < b 



(7.6) 



P[B t + /it hits 6 before a\B = 0] 



-2(b-a)fi 



when /i ^ 0, and equals | when /x = 0. 

A scale function for the diffusion, a function, S say, which satisfies u(x) = {S(x) 
S(a)}/{S(b) — S(a)}, may be deduced from ( |7.5|) to be 



(7.7) 



5(a;) 



2 / exp{-2 f fi(z)}dy, 
Jo Jo 



the scalar 2 being inserted for later simplicity. 

For the step function n considered here, the above may be integrated out for all x. 
However, our interests here require S only for integer values of x = n, and in this case, 
S(0) = and 



(7- 



u(x) 



( n— 1 k 

^r(^ fe )exp{-2^/i j } ifn>0, 

3=0 



k=0 



-1 -1 

^Jr(/i fc ) exp{2 ^2 N} if n < °- 



k=n 



The desired transition probabilities ^ follow directly now from ( [7.81 ). It suffices to con- 
sider j = 0. Since p = u(0) when 6 = 1 = —a, (|7.8|) implies that 



(7.9) 



Po = p{fJ>o,fi>- 



g(0)-g(-l) 

S(l) - £(-1) ~ rMe-^o+r^x) 



r(/i_ 



where r(-u) = (e 2u — l)/u for w 7^ and r(0) = 2. Note that p(0, 0) = | as required for 
standard Brownian motion. Using the fact that r(u) exp (— 2u) = r(—u), we summarize this 
as follows: 

Lemma 7.1 For the diffusion defined by (\7.^) , the transition probabilities of the embedded 
random walk on the integers that are defined by ( \7.3j ) are given by 



(7.10) Pj 
forj = 0,±l, ±2,- ■ 



^•(e 2 ^- 1 - 1) 



^■(e 2 ^-! - 1) + - e" 2 ^) r(/i i _i) + r(-fij) 
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It is clear that the recurrence or transience of this diffusion agrees with that of the embed- 
ded random walk. By Section 6, this in turn depends upon the quotients, pip 2 ■ ■ ■ Pk/ Q.1Q.2 • • 'Qk- 
From (fTTTOD , 



(7.11) 

Then for any k > 1, 
(7.12) 



P± = y(^j-l) c 2 W 

<?j r (/ i i) 



TT^ = r(fMo) 



exp 



j'=i 



with a similar expression for negative indices. Substitution of these into ( |3.1|) and (|3] 
would then determine recurrence or not. 

It is of interest to point out that the pj's may be evaluated directly from (|7.7| ) without 
finding the scale function. To see this, set b = 1 = —a and let x G [—1, 1]. By partitioning 
the event [W. hits 1 before — 1] according to hitting or not before 1 and —1, the Markov 
property and Anderson's result ( |7.6| ) yield 



(7.13) 



\ _ g-2x/ii 
1 - P.- 2 M1 



-2m 



+ 



-2m 



-u(0) ifx>0, 



u(x) = < 



-2(l-x)n3 



1 - e~ 2 ^ 



-u(0) 



if x < 0. 



It therefore remains to derive u(0). 

For a G (0, 1] let v(a) denote the value of u(0) when the barriers at ±1 are replaced by 
±<x That is, v(a) is the probability of hitting a before —a given the process starts at zero. 
By partitioning the event of hitting 1 before —1 according to which of a or —a is hit first, 
one obtains 

(7.14) u(0) = v{\) = v{a)u{a) + [1 - v(a)]u(-a). 

Upon substitution of 1 3|) and then solving for v(l) one obtains 

/(«) 



(7.15) 
with 



v(l) 



/(«) 



l/»(a) + /(a)-l 
1 - e - 2a «)(l-e~ 2 ' 1 - 



Observe that the limit of f(a) as a \ is 



/(0+) 



Mo 



1 = t-(M-i) 

e -2 W r(-/i )' 
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By scaling, p{a) is the same as v(l), but with /i 2 replaced by a/ii, a/i 2 . Thus one concludes 
that p(0+) = ~. Substitution of these limits into the right hand side of ( |7.13|) leads to 

(7.16) v(l) = p(jio, fi-i) = po '(/'-^ 



r(^2) +r(-Ho) 
as desired. 

For our interests here, consider the mod m shift diffusions in which fij = fii whenever 
j = I mod m. In this case, Lemma |7.1| implies that 

m— 1 ( m—1 

(7.17) p m =H^ = exphJ2H 

3=0 ^ I j=0 



so that by Lemma |3.1| , the embedded mod m random walk, and hence the mod m shift 
diffusion, is recurrent, transient toward +oo or transient toward — oo according as 

m— 1 

(7.18) ^^. = 0, >0 or < 0. 

j=0 



Then, by Lemma 3.2, the constant probability of "success" on Z m is 



m—l 



(7.19) z4 = l/ ^l + expj-2^^ 

Observe that the walk is fair {p* m = |) if and only if /x + yUi + ■ • • + /i m -i = 0. 

If one is given the p/s, one may solve the system of equations given by (|7.10|) for j = 
0,1,... , m— 1 to find the shift rates /xo, • • • , A*rn-i f° r the associated mod m shift diffusion. 
For example, for the random walk related to Game B of Harmer and Abbott (1999a) in 
which m = 3, p = 1/10 and p\ = p 2 = 3/4, the drift rates are 

^ = -.687032, /ix = 2.748128, /i 2 = -2.06109. 

Note that these are proportional to (—1,4,-3). In fact, for any fair game G(3,p,p'), the 
associated drift rates (/i , ^2) ar e equal to / u 1 (— g, 1, —p) as given by 

Theorem 7.2 // i7ie transition probabilities, Po,Pi,P2, of a recurrent mod(m) shift diffu- 
sion are known, the associated drift rates may be determined uniquely as follows: 

i) . If each pi equals 1/2, then each /j = 0; 

ii) . If exactly one of the pi's, say p 2 , is equal to 1/2 then (/i , /jli, /i 2 ) = (0, x, — x) with x 
being the solution of po/qo = (1 — e~ 2x )/2x; 
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iii). If none ofthepi's are equal to 1/2, then 

(/io,A*i,^) = (^lnw)(-(l-0),l,-0)) 

in which 9 = (1 — qi/pi)/(l — Po/lo) an d w = e 2w zs i/ie positive solution other than 1 of 
the equation 

(7.20) aw-w e + {l-a) = 
where a = (92/^2)^- 

Proof. We prove case iii) first. Write x = \i\ and y = Set a = piP2/qiq2 and 6 = ^2/^2, 
neither of which equals 1. By ([7.1 1|) the equations to be solved are 

(7.21) a = r(x + y)/r(y) and b = r(x)e 2y /r(y) = r(x)/r(—y). 
Observe that 

x + y e 2 ( x +y) — 1 ^ e 2x — 1 2y \ + x ^ 

y e 2y — 1 e 2y — 1 y ' 

noting that the arguments of r are not zero in this case. Hence, if we set u = x + y, we must 
have x = cu and y = (1 — c)u with c = (a — l)/(b — 1). Substituion into the second equation 
of ([7.2 1|) gives 

1-c e 2cu -l 

By setting w = e 2a; = e 2cn this equation becomes 

1-- =0. 



which completes the proof of iii). 

Case i) is clear. For ii), the constant b above is equal to 1. Since r is an increasing 
function, this means x = —y. The first equation then becomes a = r(0)/r(y) = 2/r(y) 
which is equivalent to the equation given in the statement of case ii). □ 

When p is rational, the equation ( |7.2(J| ) of Theorem becomes a polynomial. Here are 
two other examples: For the fair game (2(3,2/3, 1/5), 9 = 2/3 and a = 6/2. The equation 
that determines /ii = | log if is by (I7.20Q , w — 3w 2 ^ 3 + 2 = 0. Upon setting z = if 1 / 3 , the 
equation becomes z 3 — 3z 2 + 2 = 0, or, equivalently, after factoring out z = 1, z 2 — 2z — 2 = 0. 
Its desired positive solution is z = 1 + so that fi\ = (3/2) log(l + v3). This implies by 
Theorem [77^ that the drift rates are 

H Q = -.502526, /i! = 1.507579, /i 2 = -1.005053. 

For the fair Parrondo game G(3,4/5, 1/17), 9 = 4/5 and a = 1/5 so that (|7.20|) becomes 
w — 5u> 4 / 5 + 4 = 0. With z = w 1 ^ 5 , this becomes, after factoring out z = 1, z 4 — Az 3 — 
Az 2 — Az — 4 = 0. The unique positive root (by Mathematica) is z = 4.99357 so that 
\i\ = (5/2) logz = 4.020378 so that by Theorem the drift rates are 

IMi = -.804076, m = 4.020378, // 2 = -3.216302. 
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A Results about stationary probabilities of Markov 
Chains 



We begin with a 1934 result of G. Mihoc that expresses stationary probabilities of a finite 
state Markov chain in terms of cofactors: See Frechet (1952), pp. 114-6. (Mihoc's original 
paper was in Romanian, and Frechet elaborated upon it in his 1938 first edition of the cited 
reference.) Let P be any k x k stochastic matrix. For any j G {1,2,... , k}, the following 
two determinants are equal, since the matrix in the second is obtained from that in the first 
by replacing the jth column with the sum of all columns: For < s < 1, 



A(s) := Isl- 



and so 



(A.l) 



s -pn 

-P21 
-Pkl 



s/XS—\ 



— Pl2 

S -P22 



-Plj-l 



Plj+U 



-Plk 



S ~Pkk 



1 - Pll -Pl2 1 
1 



-Plk 



-Pkl 



1 l-Pkk 



Observe that the left hand side does not depend upon j. Hence the right hand side evaluated 
by expanding along the j-th column does not depend upon j. That is, if A y - denotes the 
(i, j)-th cofactor of A(l), 



(A.2) 



A. 



Ay + ■ ■ ■ + A 



Ay 



A.l, 



J = 1,2, 



k. 



On the other hand, direct evaluation of A(l) = |I — P| by expansion along the j-th column 
gives 

k 

A(l) = ^Ay((J <j -p ii ). 



i=l 



Since A(l) = this shows that 
(A.3) 



A 



i=l 



If 1 is a simple root of A(s) =0, (when the corresponding Markov chain has a single recurrent 
class) the derivative in (|A.1[ ) is non-zero so that the common sums A.j are non-zero. In this 
case, (|A.3| ) implies that for each j, (Ay, A 2 j, . . . , A k j)/ A.i is a solution in x of 



(A.4) 



x = xP, 



E 

i=l 



1. 
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Thus if P is also such that ( |A.4| ) has a unique solution, which is the case of P n converging as 
n — > oo, these solutions must all agree (with the common row elements of that limit) so that 
the numbers A^/A.j = IT, say, do not depend upon j. Equivalently, the cofactors of I — P 
form a matrix all of which columns are equal whenever ( |A.4| ) has a unique solution. (The 
reader will note the relationship to Cramer's rule for solving simultaneous linear equtions.) 

Even when P n does not converge the columns of cofactors are still all the same as long as 
the corresponding Markov chain has only one recurrent class. Here is the case of a periodic 
chain of period 2 which is neede for this paper. 

Suppose the stochastic matrix P in the above discussion is a periodic matrix C of period 
2 of the form given in ( |5.8| ), namely, 



C 



A 
B 



in which A is r x t and B is t x r with k — r + t. (The non-square nature of A and B makes 
this slightly different from the C of ( |5.7| ).) Then, 



(A.5) A(s) = \sl k - C| 



Sl r -A 

-B sl t 



It is known (e.g., Rao (1973), p. 32) that determinants of this form can be evaluated in 
two ways giving 



Therefore, for u = s 2 , 



and so 



A(s) = s\sl t - s~ 1 BA\ = s\sl r - s~ l AB\. 



A( v / u) = \ul t - BA\ = \ul r - AB\ 



, \u\-BA\ , \uI r -AB\ 
lim = hm 



u/l U — 1 u/\ U — 1 



By ( [Op and flE^) above, this means that the common column sum of cofactors of \ — BA 



is equal to that of the column sums of cofactors of I r — AB. Moreover, since 

, A(s) , A(Ju) 
lim = 2 lim x 



.s 



/\ S — 1 u/l U - 1 



each of these column sums is exactly half of the equal column sums of cofactors of I*. — C. 

It is possible also to show that the set of diagonal cofactors of I — C is made up of the 
diagonal cofactors of It — BA and I r — AB. Write a and (3 for the first row and column of 
A and B, respectively, so that 

A=(%) wdb=((3 1 B*). 
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Then the cof actor An of — C is 

A n = 

But since 

I r - AB 



I r _i A* 
B* I t 



\lr-l - A*B* 



1 - a(3 -aB* 

-A* 13 I r _i - A*B* 



it is clear that its first diagonal cofactor is also |I r _i — A*B*\. On the other hand if we use 
instead the partitioning 



A= (a*, A* 



B 



(3* 



in which a* and f3* are the first column and first row of A and B, respectively, then 

(a*A**) 
If 



Ifc-C 




Therefore, the (r + l)-th diagonal cofactor of I — C is 

- B**A**\. 



Ir A** 
B** 



But since 



It-BA 



1 - (3*a* -/3*A** 
-B**a* I t _! - B*A* 



its first diagonal cofactor is \It-i ~ B* A* | as well. 

By cyclically permuting the first j columns and rows when 1 < j < r, or the (r + l)-th 
through j-th columns and rows when r < j < k, the above arguments prove that the first 
r diagonal cofactors of — C are those of I r — AB and the last t of them are the diagonal 
cofactors of It — BA. 

In view of the above results, the stationary probability vectors 8 and p for AB and BA, 
respectively, that were introduced for (|5.8|) may be expressed in the notation of Section 4 as 



(A.6) 



"(7 



lrrt) 



, 7 fcm ) and p 



1-r 



"(7 



fc+i 



In particular, this verifies the equivalence of ( |5.7| ) and (|5.9|), showing that ( p.9|) applies for 
all m, whether even or odd. 

Acknowledgement: The author is grateful to Derek Abbott and Greg Harmer for their 
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